Numerous algebraic modeling methods have been proposed in efforts to organize the properties of complex systems in order to control and/or predict their behavior. Examples of applications of modeling techniques to complex systems include economic modeling of securities, inventories, cash flow, sales, and marketing, manufacturing and systems control, and scientific applications to spectral analysis. In many such systems, while the real number of underlying variables that describe the system properties may be small, these variables are unknown. Because of the complexity of the data and the presence of sampling errors, the model can end up with too many parameters, the quantity of which can equal, or even exceed, the number of data points. This is a common problem in nonparametric analysis, where using too many parameters leads to large statistical uncertainty and biases in the derived model parameters, and in the correlations among them.
In the area of financial prediction, numerous methods have been proposed that are based upon covariance matrix models. For example, in U.S. Pat. No. 5,444,819, of Negishi, an economic phenomenon predicting and analyzing system using a neural network is described. Learning data is input into the network, including past trends, patterns of variations, and the objective economic phenomenon corresponding to the past data. The hidden layer acts as a number of covariance matrices, categorizing the data in an attempt to identify a small number of principal variants or components, ideally reducing the number of variants to be considered in the prediction of moving averages.
This neural network of Negishi is an attempt at applying the principle of “minimum complexity”, also called “algorithmic information content.” This principle is a manifestation of Ockham's razor—to minimize the number of parameter required to fit the system. Minimum complexity enables an efficient representation of the complex system and is the best way to separate a signal from noise. (For purposes of this application “signal” means the desired information, which can be financial data or other information to be extracted from an input containing an excess of information, much of which can be considered superfluous background noise.) If the signal can be adequately represented by a minimum of P parameters, addition of another parameter only serves to introduce artifacts by fitting the noise. Conversely, the removal of too many parameters can result in an improper representation of the system, since adequate fitting of a model to the system requires a minimum of P parameters.
While minimum complexity has a clear theoretical advantage, it can be computationally intensive, making it difficult to reach a conclusion in a period of time that would permit practical application, unless the parameterization of the system in known in advance. A representation of a system requires a model language that decomposes it into smaller units, and one must choose between a vast number of languages, i.e., means for expressing the algorithm. Even after a language is chosen, the set of all possible parameterizations with that language can become too large to search practically. For example, consider modeling the covariance matrix of a system of N variables with P parameters, such as discussed above relative to the Negishi patent. Standard estimates assume that all the elements of the covariance matrix are significant, i.e., each variable is correlated with every other variable. This gives N(N+1)/2 independent elements of the covariance matrix (after accounting for the symmetry of the covariance matrix). A minimum complexity model seeks to represent these N(N+1)/2 numbers by a much smaller number P of parameters. One simple approach would be to set to zero all but P of the N(N+1)/2 elements. However, the choice of P elements among N(N+1)/2 is a combinatorially large problem and an exhaustive evaluation of all of the possibilities is not practical. In addition, the covariance matrix must be positive definite, and this constraint further restricts the possible parameterizations. It is clear therefore, that a practical method is needed which will find a minimum-complexity model without requiring an exhaustive search of all possible parameterizations.
Others in the field have proposed prediction and risk assessment techniques based using covariance matrices, with some developing relatively complex models with a large number of variables, thus producing a computationally-intensive model. See, e.g., Tang, “The Intertemporal Stability of the Covariance and Correlation Matrices of Hong Kong Stock Returns”, Applied Financial Economics, 8:4:359-65; Nawrocki, “Portfolio Analysis with a Large Universe of Assets”, Applied Economics, 28:9:1191-98. Others have proposed models in which the number of parameters is so small that one must be concerned about the accuracy of the representation of the system. See, e.g., Hilliard and Jordan, “Measuring Risk in Fixed Payment Securities: An Empirical Test of the Structured Full Rank Covariance Matrix”, Journal of Financial and Quantitative Analysis, September 1991.
In related application Ser. No. 333,172, the inventors disclose a signal and image reconstruction method which utilizes the minimum complexity principle, in which the method adapts itself to the distribution of information content in the image or signal. Since a minimum complexity model more critically fits the image to the data, the parameters of the image are more accurately determined since a larger fraction of the data is used to determine each one. For the same reason, a minimum complexity model does not show signal-correlated residuals, and hence provides unbiased source strength measurements to a precision limited only by the theoretical limits set by the noise statistics of the data. In addition, since the image is constructed from a minimum complexity model, spurious (i.e., artificial or numerically created) sources are eliminated. This is because a minimum complexity model only has sufficient parameters to describe the structures that are required by the data and has none left over with which to create false sources. These fundamental parameters are known as Pixon™ elements, which are also described in related U.S. Pat. No. 5,912,993. Finally, because the method builds a critical model and eliminates background noise, it can achieve greater spatial resolution than competing methods and detect fainter signal sources that would otherwise be hidden by background noise.
It would be desirable to apply the methods of minimum complexity to a method for prediction of behavior of complex systems where the behavior can be modeled algebraically, and where the computation time is appropriate for practical applications.